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Abstract 

There was a time when virtually all students pursuing 
the master’s degree were required to submit to and 
successfully complete comprehensive examinations, most 
of which featured both written and oral elements. In this 
more traditional period, the acquisition of the master’s 
degree was in large part a qualifying ritual for admission 
to a Ph.D. program which, in turn, assumed a subsequent 
professional life in the academy. With the gradual shift in 
the character of the degree from the purely academic to 
the at least quasi-professional, alternative approaches to 
evaluating student performance are emerging as exit 
requirements for the master’s degree. This paper reports 
the results of a survey, featuring both quantitative and 
qualitative components, designed to investigate the status 
of the traditional comprehensive examination, as well as 
the nature and distribution of alternative assessment 
methods. 

Introduction 

Much of what occurs in colleges and universities, from 
curriculum to governance structure, proceeds on a 
normative assumption, i.e., that there are principles which 
are fundamental to the practice under investigation. It is 
assumed, for example, that students will be required to 
follow programs of study which are carefully crafted to 
adhere to what is considered “normal” for the field and to 
the standards and expectations of accrediting bodies. 
Thus, if one were to pick up a college catalogue in order 
to investigate which courses are required for the MA in 
applied mathematics, it would be surprising if many were 
to deviate from the commonly prescribed curricular 
substance and sequence. 

The issue of the comprehensive examination for the 
master’s degree appears to conform to this phenomenon. 



Singletary (1 999) observed that “the extent of reliance on 
master’s comprehensive examinations is undocumented 
but believed to be strong,” suggesting that the requirement 
that candidates for the master’s degree demonstrate 
competence by successfully passing an examination 
remains standard. 

First, we needed to verify Singleton’s contention that at 
some time in the past the reliance on a comprehensive 
examination was “strong.” When we surveyed 50 graduate 
catalogs for the 1 983-1 984 academic year we found that 
most graduate schools required a comprehensive 
examination — either written or oral — at the master’s level, 
with the exception of students doing a thesis. In those 
cases, the defense of the thesis substituted for a 
comprehensive exam (and may, in fact, have been 
considered equivalent.) Of the catalogs surveyed, 86% 
explicitly mention the comprehensive exam. Those that 
fail to mention such an exam may still have required one 
given the culture of the time and the implicit assumption 
that graduate students were supposed to take “comps.” 

With the master’s degree considered preparation for 
the Ph.D., master’s candidates were expected — ^just like 
doctoral students — to write theses and take grueling written 
examinations. If one could handle these challenges at the 
master’s level, presumably one was a reasonably good 
candidate for a Ph.D. program. And, if one failed to 
complete the doctorate, one at least had an academically 
legitimate consolation prize. 

A search of several databases which archive academic 
literature seems to confirm the suggestion that the 
comprehensive exam retains a level of ubiquity. A search 
in Academic Search Elite, which provides full text articles 
from 1,530 periodicals dating back to 1990 and offers 
abstracts and indices from 2,720 periodicals back to 
1984, returned only 16 references under the subject 
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“comprehensive examination,” 1 1 of which were reiated to 
standardized testing in the K-1 2 sector. Of the remaining 
five references, two concerned study guides for an 
engineering examination, one the master’s examination 
in economics, one comprehensive examinations at the 
doctorai ievei, and one the reiationship between 
comprehensive and iicensing examinations for the doctorai 
degree in psychoiogy. When the search string was 
changed to “comprehensive assessment,” 15 references 
were returned, 13 of which focused on K-1 2 concerns. 
The remaining two invoived articies on a comprehensive 
heaith care network and on an anaiysis of the dissemination 
of pubiic information conducted by the Nationai Councii of 
Libraries. 

Expanding the search banks to inciude the Education 
Resources information Center (ERiC) and Ebsco databases 
in addition to Academic Search Eiite returned more of the 
same pattern. Whiie the number of returns was increased 
(32 returns to “comprehensive examination” and 50 to 
“comprehensive assessment”), the vast majority 
maintained the focus on standardized testing at the K-1 2 
ievei. Of the eight articies with any reiationship to higher 
education, seven examined subject-specific examinations, 
e.g., the reiationship between a comprehensive 
examination in nursing and subsequent performance on 
the Nationai Councii Licensure Examination (NCLEX), 
and the other the impiementation of a portfoiio-based 
assessment in an educationai administration program. 
Searches using the string “master’s” + “comprehensive” + 
“examination” returned no citations. 

One might assume, from the apparent scarcity of 
citations, that Singietary’s (1999) observation is correct, 
it seems piausibie, however, to at ieast attempt to confirm 
it in iight of emerging chaiienges to the orthodoxy impiicit 
in assessment rhetoric at both the undergraduate and 
graduate ieveis. This discourse has been accompanied 
by a variety of progressive approaches, among them an 
increasing acceptance of quaiitative measures, which 
some characterize as iess authoritative but more 
iiiuminating than the quantitative measures which have 
dominated the fieid for iiteraiiy decades. Among the more 
visibie of these approaches is portfoiio assessment. The 
portfoiio, whiie stiii presentiy used primariiy in conjunction 
with some more conventionai evaiuation methods (e.g., 
iicensing examinations, certification tests, etc.), has been 
gaining the support of both students and facuity across 
discipiines, and has had a significant impact on teacher 
education programs (Dariing-Hammond et ai., 1 995; Hoimes 
Group, 1986; Moss, 1997; Nationai Board, 1989; Schon, 
1987;. 

Severai graduate programs in our own university have 
been among those making the transition from the more 
conventionai comprehensive examination to aiternative 
forms of assessment for master’s degree candidates. 
Compietion of the master’s degree in educationai 



administration and in eiementary and secondary education 
are among those which have undergone a transformation 
in their assessment practices, requiring the deveiopment 
of a portfoiio which demonstrates students’ competencies 
as they reiate to the standards of the appropriate 
accrediting body, it was this shift, in both our own programs 
and in others of which we are aware, which prompted the 
current study in order to discern whether the transformation 
of assessment practices is, in fact, widespread, or whether 
reiiance on those more traditionai measures remain as 
strong as Singietary (1999) impiies. 

Theoretical Framework for the Study 

Recent decades have seen a significant increase in 
chaiienges to what is variousiy described as the modernist 
or iogicai positivist approach to conducting research from 
methods which are more cioseiy aiigned with postmodernist 
or postpositivist thinking. Modernist/positivist methods, 
most cioseiy identified with quantitative research, generaiiy 
take for granted a kind of binary iogic that assumes the 
phenomenon in question either does or doesn’t conform to 
preconceived hypotheses. Research questions, thus, can 
be neatiy divided into categories with discrete and mutuaiiy 
exciusive responses. 

Peopie with a more postmodern approach to research, 
on the other hand, stress the importance of context and 
perspective, questioning the appropriateness of attempting 
to reduce compiex questions to simpie yes/no proportions. 
Many wonder if it is even possibie to undertake a scientific, 
objective study in a fieid which is characterized as 
“uncertain, unstabie, compiex, and often unique” 
(Cunningham and Cordeiro, 2000, p. 5). We find this 
question iegitimate. We aiso, however, find it unnecessariiy 
iimiting to reject traditionai quantitative research methods 
on the ground that they are insufficientiy nuanced to be of 
any use in a iess than tightiy structured study. We share 
with Griffiths (1995) an interest in a research agenda 
which is probiem-focused and which adopts a theoreticaiiy 
diverse approach. 

This study, thus, reiies on a combination of both 
quantitative method and quaiitative anaiysis. it biends 
responses to discrete questions with an examination of 
responses to open-ended questions in order to derive as 
compiete a picture as possibie of the state of the 
comprehensive assessment as the cuiminating experience 
for the acquisition of the master’s degree in the institutions 
surveyed, it is an attempt to address what we see as two 
reiated issues: 1) the nature of the comprehensive 
assessment and 2) reasons for any changes institutions 
have made in their poiicies concerning that assessment. 

Format of the Study and Methods of Analysis 

A survey instrument featuring 11 questions, with a 
request for respondents to eiaborate on certain responses, 
was sent to 450 members of the Councii of Graduate 
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Schools. In order to generate the largest possible number 
of completed surveys, respondents were invited to choose 
whether to answer questions via the Internet, where we 
established a Web site for the survey, or to complete and 
return the paper copy. Of the 1 1 questions, two were 
identifiers (i e-, whether the respondent represented a 
public or private institution and whether the institution is 
research extensive, research intensive, master’s I or 
master’s II), and one question asked whether the respondent 
would agree to a follow-up interview. Eight focused on the 
research question itself. 

Of the 450 surveys mailed, 1 1 5 were returned by mail 
and 38 via the Internet, for a total of 1 53 (a response rate 
of 34%). Following standard quantitative research 
measures, such a response rate fails to conform to 
expectations for generalizability. Given the interest colleges 
and universities maintain in assessment practices, however, 
we thought it useful to proceed with a quantitative analysis 
of the data nonetheless. If the results of the research are 
to be used to inform decision making at the administrative 
levels in any way, questions of validity and reliability may 
be raised as a result of the return rate. In this study, we 
establish frequencies of responses and percentages of 
totals in order to quantitatively array the data.’ Validity 
can be looked at as the responsiveness compared to the 
population surveyed, based on certain characteristics such 
as institutional type and geographical location. 

It is in the explanations for certain responses, however, 
that much of the valuable and interesting data lie - in 
narrative which cannot be reduced to frequencies and 
percentages. For that reason, our analysis of the narrative 
data, and of the comments of respondents who agreed to 
subsequent interviews concerning the subject, is conducted 
from a qualitative perspective. 

Phenomenological analysis, the qualitative approach 
we have chosen, involves a mutual relationship, one in 
which the interviewer and the interviewee(s) are equal 
participants or “co-researchers” (Nelson, 1989) in the 
effort to gain some insight. Rather than adhering to a 
format of carefully structured questions, the protocol 
features an open-ended dialogue wherein the participants 
can generate data together “in a social context where 
[they] can consider their own views in the context of 
others” (Patton, 1980, p. 335). It is a process we think 
appropriate for examining more closely this phenomenon 
in which we share an interest with our peers. 



Lanigan (1988) describes phenomenological analysis 
as conceived by Merleau-Ponty, widely regarded as the 
seminal thinker in the field, as a three-step process 
requiring description, reduction and interpretation. Each 
step informs the next in a progressive fashion to yield what 
Lanigan calls “a systemic completeness” (p. 173). 

• The descriptive phase involves a careful reading of or 
“listening to” the data absent any preconceived ideas 
or classifications. The intent at this stage of the 
analysis is to allow for “the widest possible number” 
of broad themes to emerge (Nelson, 1989, p. 232). 

• The second step, reduction, extracts from the 
emerging themes those which can be seen as 
constitutive. The aim of reduction, as explained by 
Cooks and Descutner (1994), is “articulating] ... a 
‘pattern of experience’ expressed through the essential 
elements of the phenomenon under investigation” (p. 
255). 

• Phenomenological interpretation, then, requires the 
examination of the primary themes which emerge to 
discern those which effectively make explicit (in this 
case, the current status of the comprehensive 
examination) what had formerly been implicit (the 
assumption that its use continues to be widespread). 

In short, we attempt to analyze the data from both a 
discrete and a holistic perspective, although a more 
complete analysis will occur subsequent to the follow-up 
interviews. After breaking down the percentages and 
frequencies where appropriate, we turn to the narrative 
data (examined in the description phase), reduce it to 
some primary themes, which we then re-examine in an 
attempt to determine which clarify in a useful or meaningful 
way our understanding of the state of the comprehensive 
examination. Having described the data-gathering 
procedure, we now turn to the handling of the collected 
data. 

Analysis 

The survey questions involving content (as opposed to 
classification of institutions responding), first concerned 
whether there exists an institutional policy requiring some 
kind of comprehensive assessment experience for all 
master’s degree students. The majority of respondents 
responded positively, as is demonstrated in Table 1 below. 



Table 1 



QUESTION #1 


# of responses 


YES 


NO 


Does institutional policy require some kind of a 




115 


37 


comprehensive assessment/capstone 


152 






experience for all master's degree students? 




76% 


34% 
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Table 2 

QUESTION #2 

If "no," what is the reason? 

QUESTION #3 

What type(s) of comprehensive assessment/capstone experiences (e.g., thesis, oral exam, 
portfolio assessment, capstone project) are typically given in the following program areas: 
business, science, fine arts, liberal arts, nursing/allied health, journalism, education, other? 



While the comprehensive/capstone assessment 
remains the standard of good practice at the master’s 
level, a university-wide policy for all programs to require a 
capstone experience is not as universal as it apparently 
once was. This appears to reflect the increasing 
professionalization of the master’s degree and the 
increasingly part-time nature of graduate students. This 
tendency may become more pronounced with the growing 
success of the Sloan Foundation initiative to 
professionalize the M.S. in science. 

Respondents reported that individual departments, 
programs, schools or colleges within the institution have 
fairly broad latitude as to the form such a comprehensive 
assessment will take. While some respondents cited the 
traditional thesis coupled with a comprehensive 
examination involving both written and oral components, 
they were few in number. Among the alternatives mentioned 
as exit requirements were research projects, research 
courses or symposia, capstone courses, creative projects 
or performances, exhibitions, internships, practice or 
clinicals, and portfolios. 

If ever the comprehensive examination predominated in 
MBA and other business programs, it appears to have 
been surpassed by capstone courses and/or capstone 
projects which emanate from those courses. Although 
we did not inquire about the nature of capstone projects 
in business programs, we speculate that they are related 
to real world problems — either problems which have 
recently been “solved” and now stand as case studies or 
problems with which businesses have asked our colleges 
and universities to help them deal. 

In science, despite the Sloan Foundation’s recent 
initiative, the thesis rather than the comprehensive exam 



dominates the responses. Whether theoretical or practical, 
the thesis and its oral defense appears designed to give 
the student an intimate experience with the research 
methods he or she, presumably, will be using either in “the 
real world” or in a doctoral program (the thesis serving as 
practice for the dissertation). 

The fine arts, reasonably enough, primarily offer students 
a supervised showcase for their talents. Although it may 
be called a “comp,” in fact in many cases it is an exhibit 
or performance and it is unlikely the faculty would let the 
student get this far without the expectation the student 
would graduate. 

The liberal arts utilize the traditional written 
comprehensive examination, but the thesis appears to be 
a more popular option. The capstone course and the 
capstone project have made their appearance and augur 
the presence of viable alternatives to the written comps 
and thesis. 

In nursing and the health professions, clinical 
experiences such as internship and practicum are frequently 
cited. A research element, however, remains strong 
whether through the research report, master’s paper, 
special project or thesis. One university listed a “scholarly 
inquiry endeavor.” 

The professional project in journalism is offered as an 
alternative to both the thesis and the comprehensive 
examination. In education, portfolio assessment has not 
yet replaced the comprehensive exam entirely, but there 
is growing respect for a process which emphasizes student 
products which evolve over time and reflect preparedness 
to enter the teaching profession. 

Among the other major options is the final project, 
typically found in engineering and information systems/ 



Table 3 



QUESTIQN #4 


# of responses 


YES 


NQ 


Is an internship allowed to be used as the 




39 


105 


comprehensive assessment/capstone 


144 






experience in any program? 




27% 


73% 



4 



IR Applications, Number 8, The Changing Nature ... 



Table 4 



QUESTION #5 


# of responses 


YES 


NO 


Is the comprehensive assessment/capstone experience 




23 


121 


used to validate outdated coursework (i.e., classes which 


144 






are older than the prescribed time limit)? 




16% 


84% 



management systems programs. According to one 
correspondent, the final (or comprehensive) project, by 
involving the application of management, engineering and 
scientific techniques, offers “a good assessment of a 
student’s ability to collect information and data about a 
particular problem, analyze this material, synthesize a 
solution, invoke the solution, and effectively communicate 
this process in both an oral and written format.” 

In summary, the type of closure experience which 
encouraged the student to cram for a multi-hour written 
examination no longer holds the hegemony it once did. 
Instead, a host of options is now available. 

While most respondents indicated that the internship in 
and of itself is insufficient to serve as the culminating 
experience, more than one-third reported that their 
institutions do accept such arrangements. Internships 
have been included, of course, as an element in 
programmatic requirements for some time. Their emergence 
as the sole comprehensive experience, however, may 
come as a surprise to some. 

Programs which allow the internship to replace the 
comprehensive exam presumably do so because they 
believe the student’s performance in the internship reflects 
all that he or she has learned in the didactic courses, as 
well as the student’s ability to use what he or she has 
learned in an appropriate and effective manner in practice. 
The proof of graduation is in the pudding of the internship. 

The fields in which internships were implemented provide 
some interesting information as well. It might be expected, 
for example, that internships in such applied fields as 
business, psychology, counseling or allied health would 
be common. Survey participants, however, also reported 
the presence of internships in fine arts, physics, history, 
and geography. 

The subject of using the comprehensive assessment/ 
capstone experience to validate outdated coursework. 
Question 5, produced the most significant figures. As 
demonstrated in Table 4, an overwhelming number of 



respondents reported that their institutions do not permit 
such a practice. Because an explanation was invited from 
only respondents who indicated that their institutions 
allow validation via the comprehensive assessment, we 
can only speculate as to the reasons for non-acceptance. 
It seems reasonable to assume, however, that most 
institutions conceive of the processes as distinct. A 
revalidation exam is designed to demonstrate both retention 
of course content and in-depth currency of subject matter 
within a particular course, whereas a comprehensive 
assessment allows students to demonstrate their ability 
to synthesize, integrate and interrelate what they learned 
in a broad range of classes. The comprehensive 
assessment would not necessarily cover all significant 
content in every outdated individual class. 

Respondents who reported that their institutions do 
accept the substitution of the comprehensive assessment 
for outdated coursework were invited to elaborate on their 
answers, but most addressed issues of specificity as 
opposed to justification. They wanted to clarify, for example, 
that only a traditional comprehensive examination would 
be acceptable, as opposed to an alternative form of 
assessment. Others mentioned that there are limitations 
related to the number of courses which could be validated 
(a maximum of two courses was the norm), as well as 
student performance on the assessment itself (e.g., a 
minimum score of 75%). One respondent noted that 
courses students have completed at non-accredited 
institutions and wish to transfer, if they reside within the 
confines of the institution’s policy concerning acceptable 
age of the coursework (e.g., completed no longer than 
seven years prior), can be so validated. 

At the master’s level most institutions do not allow 
revalidation through means of the comprehensive exam. 
Instead, they tell students to retake their outdated classes, 
update through a special exam for each individual class, 
update through independent study, or substitute other 
coursework on the students’ Plans of Study. A number. 



Table 5 



QUESTION #6 


# of responses 


ONE 


TWO 


THREE OR 
MORE 


How many faculty typically evaluate the comprehensive 


149 


11 


28 


110 


assessment/capstone experience? 


7% 


19% 


74% 
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Table 6 



QUESTION #7 


# of responses 


SOME 


NONE 


Under what circumstances, if any, do you allow students to 




38 


91 


submit or participate in team or collaborative 


129 






comprehensive assessment/capstone experiences? 




29% 


71% 



such as Marshall, are now turning to portfolio assessment 
as an additional option. 

Question 6 also produced some fairly definitive 
responses. The norm for number of faculty comprising 
MA comprehensive assessment committees has been 
three for many years, and remains the most popular 
among our respondents. With more than one-quarter of 
the survey participants reporting fewer than three, however, 
it should be noted that the judgment as to whether a 
student has successfully completed the comprehensive 
assessment/capstone course often rests with only one or 
two faculty members. 

Perhaps the reduction constitutes a concession to the 
reality that, in the face of increasing numbers of part-time 
faculty, too much work is being required of too few full- 
time faculty members. That is, if a student’s performance 
is to be the product of a majority vote, it may be perceived 
a pointless allocation of a scarce resource to require a 
third reader if the first two agree. Whatever the reason, 
the concept of a three-person committee at the master’s 
level is no longer an absolute rule. 

Turning to question 7, the current mantra is that 
interdisciplinary teamwork results in better science and 
reflects the way of the “real world,” whether it is business, 
education, nursing or whatever works. Nonetheless, the 
majority of deans heaved a collective gasp of horror at the 
idea of students’ participating in a group comprehensive 
assessment. While acknowledging that there is a role for 
collaboration at a certain level, they believed that a 
judgment, ultimately, must be made about each individual 
student’s success in having met the requirements for 
graduation. They feared that team projects reward the 
weak and penalize the superior students. For them, it 
appeared a matter of fairness. 

This particular question invited elaboration from all 
respondents, and provided additional insight. Among 
respondents who indicated the existence of collaborative 
projects, one raised a potentially problematic issue: the 
incursion of technology into virtually every nook and cranny 
of the academy. Explaining the institution’s choice to 



accept a collaborative project in a technology program, he 
noted that the institution’s “multimedia M.A. students do 
a team project because no one person can do a professional 
multimedia project by himself or herself.” While such a 
position is perhaps arguable (e.g., should a student who 
plans a career in multimedia have a basic operational 
grasp of multiple technologies and be able to demonstrate 
her competence across them by herself?), it is neither our 
intent nor our place to question institutional policy. 

Rather, we found this particular observation interesting 
for the questions it raises concerning the use of technology 
in the academy in general and the expectations held by 
institutions and programs (imposed in large part by 
accrediting agencies) relative to student technological 
competency. If, for example, a student in a M.Ed. program 
is required to compose a multimedia presentation 
demonstrating his mastery of National Council for 
Accreditation of Teacher Education (NCATE) standards in 
partial fulfillment of degree requirements, and he is assisted 
by another student whose technological expertise 
surpasses his own, does that render the final product a 
collaborative one? If students are required to submit their 
theses electronically and do so by acquiring the assistance 
of a technologically savvy peer or even professional, have 
they failed to conform to procedure? 

Certainly these are questions to which the answers are 
specific. If the requirement for an electronic thesis 
submission is made for reasons of institutional expedience 
rather than an attempt to assess students’ technological 
competence, one supposes the issue of who handles the 
actual submission is of little import. If, however, programs 
are implementing requirements for culminating experiences 
which involve technological elements on the ground that 
they can assess students’ competence in line with 
institutional or accrediting agency expectations for 
technological literacy, the issue could become problematic. 
How much assistance can a multimedia expert provide 
before her involvement renders the project “collaborative”? 

The number of institutions allowing team or collaborative 
comprehensive assessments does, in any case, appear 



Table 7 



QUESTION #8 

How do you prepare students for the comprehensive assessment/capstone experience? 
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to be growing. That finding is consistent with what was 
revealed in responses to the previous questions: the 
nature of the comprehensive process is changing, from its 
composition to its presentation to the number of faculty 
who evaluate it. 

A variety of methods is used to prepare students for 
their comprehensive assessment ranging from literature 
lists, providing sample (old) exams, posting topics, seminars 
designed to review content and applications, to discussions 
with (and perhaps helpful hints from) either the major professor 
or members of the student’s graduate committee. 

One thing that is clear is that the individual departments 
and programs take the lead here. The Graduate Dean’s 
Office does not claim a central role in assisting students 
at this juncture in their academic travail. 

Discussion 

With this explication of the quantitative elements of our 
study, we turn to the qualitative analysis of our data - the 
descriptive, reductive and interpretive elements. 

Description. Lanigan (1 988) reminds researchers that 
it is important at the description phase to attend to 
elements in participants’ discourse which demonstrate 
their “awareness of what [the] phenomenon is” (p. 337). 
Those involved in this study revealed a clear understanding 
of their institutions’ assessment practices. From our 
analysis of their comments, three broad themes emerged. 

First, the question of institutional policy regarding both 
the value and existence of some form of comprehensive 
assessment appears to rest on historical assumptions 
and professional recommendations. The American 
Association of University Professors (AAUP) in 1935 
recommended that the master’s degree culminate in a 
thesis and a comprehensive examination, a 
recommendation which the AAUP reiterated in 1 945 (pp. 
10-11, Conrad). 

We have witnessed more of a retreat from the thesis 
requirement than from the comprehensive examination. 
“As the master’s became less a traditional research degree, 
especially in the arts and sciences but also in some 
professional fields, many master’s programs either dropped 
the thesis requirement altogether or introduced a ‘non- 
thesis’ option [Boddy 1970]” (cited in Conrad, p. 22). 
While this appears increasingly to be true, the 
comprehensive exam has largely survived, albeit in ways 
that significantly redefine and transform the term. 

Second, the culminating or closure experiences at 
American master’s institutions are exceptionally diverse. 
It would no longer be accurate to say, for example, that 
“the written comprehensive examination is the most 
common requirement in education master’s degree 
programs” (Jenkins and Douzenis, 1 998). Specific choices 
concerning appropriate methods for assessing students’ 
competencies largely reside at the college, school or 
departmental levels. 



Third, the nature of the comprehensive assessment 
has changed significantly. As is becoming increasingly 
apparent, the goal of the comprehensive assessment at 
the master’s level is not merely to measure what the 
student has learned but, just as importantly, to demonstrate 
how he or she applies what he or she has learned. 
Various projects and portfolio assessments permit the 
student to show competency to enter a professional field — 
or to earn advancement in his or her field. Some academic 
units, to be sure, continue to require the traditional 
comprehensive examination with both written and oral 
elements. Alternative forms of evaluation, however, appear 
across the disciplinary board. This development is not 
surprising. We might have cause to worry, in fact, if 
faculties failed to take advantage of research indicating 
that a breadth of evaluative practices is more likely to elicit 
an accurate view of student performance. 

Not everyone is happy about the de-emphasis of the 
thesis. As reported by Robert M. Jackson in a December 
14, 2001 , e-mail , a number of California State University 
deans deplore this change because 

“the thesis and project are viewed as requiring a 
sustained demonstration of independent research 
skills or the creation of an educational/professional 
product in the case of a project (software, 
instructional materials, training manual, etc.). This 
is viewed as a more demanding challenge than a 
few hour exam.” 

Flowever, few academicians any longer view the 
comprehensive assessment as merely a few hour exam. 

It is not unusual that such culminating experiences as 
research symposia, internships or practice, capstone 
projects and the like be part of the student’s requirements 
for completing the master’s degree. These are fairly 
universal elements in graduate programs, and are often 
used in conjunction with the routine examination. That 
some of these alternative approaches have replaced the 
conventional comprehensive examination, however, and in 
not only a few instances, suggests the emergence of a 
significant change in assessment practice. 

Reduction. What the previous descriptions represent 
are expressions of participants’ connotative understandings 
of comprehensive assessment practices. The reduction 
step in the analytical process requires us to extract from 
those descriptions those which can be characterized as 
essential elements and formulating them into “concise 
expressions” (Nelson, 1 989, p. 235). Our analysis reduces 
these elements to two such expressions: 

• Some form of comprehensive assessment remains 
imperative in the academy. While the institutional 
representatives who responded to the survey 
acknowledged changes, some of them significant, in 
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the character of assessment practices, all expressed 
a commitment to maintaining the integrity of the 
master’s degree by ensuring that students 
demonstrate in some fashion their acquisition of the 
knowledge, skills and dispositions it represents. 

• Decisions concerning the nature of the comprehensive 
assessment should be made at the level of the 
academic unit which certifies that the student has 
mastered the requirements for the awarding of the 
degree. 

Interpretation. The final step in the phenomenological 
analysis is to examine the themes and their subsequent 
reductions in an attempt to make explicit what has 
previously been implicit. In this study, implicit feelings or 
impressions about the changes in the nature of 
comprehensive assessment were transformed into explicit 
examination of current practices. If we were to isolate, as 
Cooks and Descutner (1994) recommend, “a revelatory 
phrase” from the interviews, something we believe is the 
central premise or “the signified in the discourse” (p. 260), 
we would need to locate a statement that makes clear the 
meaning of culminating assessment for people involved in 
institutional policy. 

The respondents, as can be seen from their descriptions 
of assessment practices, did seem to share a sort of a 
pr/on understanding of the dimensions of comprehensive 
assessment, particularly as it relates to the traditional 
process. Unlike many phenomenological studies, wherein 
participants’ descriptions tend to demonstrate a kind of 
fluid, mutable character in recognition that judgments of 
the phenomenon in question are highly personal, this 
investigation is notable for the degree of consensus it 
reflects. The smallest majority response, for example, 
was 60% (on Question 7). Other majorities, as can be 
seen in the previous tables, ranged from 73% to 84%. 

The statement we believe best makes explicit our 
respondents’ understanding of the status of institutional 
policy as it relates to the comprehensive assessment is 
this: Assessment is both a normative and a variable 

enterprise; while comprehensive assessment remains a 
critical element in master’s level graduate education, its 
dimensions are quite diverse. 

Observations and Conclusions 

In reading the comments of these interviews, we find 
more than an occasional trace of postmodern thinking. 
The modernist notion of a traditional comprehensive 
examination which is both the exclusive and conclusive 
instrument for determining whether a student merits the 
awarding of the master’s degree bears little relation to 
what emerged from the discourse. Postmodernism 
accepts the existence of indeterminacy as a given in the 
contemporary culture the central characteristic of which 
is its lack of homogeneity. It is an understanding which 



confirms Cunningham and Corderio’s (2000) description of 
educational administration, the endeavor in which we are 
all involved, as a field which is characterized as “uncertain, 
unstable, complex, and often unique” (p. 7). 

The compatibility of our data with this postmodern 
perspective leads us to two observations, both grounded 
in Griffith’s (1995) claim that much contemporary research 
engages in theory-building at the expense of problem- 
solving. First, as is related to the frameworks under which 
research in educational administration is conducted, we 
find Griffith’s advocacy of theoretical pluralism to have 
merit. The data we were able to glean from the kind of 
open-ended, dialogic approaches common to qualitative 
analyses generated information which would have been 
difficult if not impossible to acquire through a conventionally 
structured survey that controls for and thus limits 
respondents’ ranges of response. The dialogic element 
enhanced the richness of the accompanying quantitative 
analysis by broadening the scope of the investigation. 

Second, in terms of problem-solving, we believe the 
study contributes to our ability to challenge the broadly 
held public perception that university expectations are out 
of touch with the professional needs of today’s students 
and their eventual employers. People who continue to 
conceive of universities as ivy-covered cloisters mired in 
the academic practices of the last century might even be 
cheered by the multitude of options available to students 
for demonstrating their competence. 

Are there conclusions which may be safely drawn from 
these observations or from the study itself? As was noted 
at the outset, the small number of respondents relative to 
both the membership of this organization and to the 
number of institutions of higher education represented 
herein necessarily mitigates that possibility. Our findings, 
thus, are more suggestive than conclusive. 

That said, however, it is reasonable to assume that 
these respondents are representative of institutions 
nationwide, both public and private, and across 
classifications. To that extent, the responses they made, 
if not generalizable, remain informative. We would be 
surprised, in fact, if a more thorough investigation yielded 
results significantly different from those reported herein. 
Still, a study of broader proportion is a recommendation 
we are inclined to make, if only to confirm the transformation 
of assessment practices our respondents indicate. 

Were such a process to be undertaken, we would also 
recommend that the researchers assist the respondents 
by taking more care when constructing survey questions 
(e.g., clarifying “capstone experience” by drawing a 
distinction between a capstone course and a capstone 
project, or defining such terms as “internship” and 
“practicum” or “symposium”). These revisions would be 
helpful to researchers as well, reducing the possibility of 
misinterpretation. Finally, now that we feel we can claim 
with reasonable assurance that a transformation of 
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assessment practice is occurring, we wouid be inclined to 
ask participants to offer their perspectives on why the 
more traditional comprehensive examination has been 
marginalized or replaced altogether in some fields which 
had previously used it as the sole exit requirement. 
Answers to that question may prove even more enlightening 
than what we found thus far. 

Conclusions 

The meat of our study is found in response to question 
3 — the type(s) of comprehensive assessment or capstone 
or closure experience conducted in master’s institutions. 
The responses were remarkably diverse. Clearly, much 
has changed since i/i/e went to graduate school. 

As recently as 1 998, Jenkins and Douzenis proclaimed 
that: “The written comprehensive examination is the most 
common requirement in education master’s degree 
programs” (1998). However, it would no longer be accurate 
to make this claim for education, or for any other academic 
discipline. 

As is increasingly apparent, the goal of the 
comprehensive assessment at the master’s level is not 
merely to measure what the student has learned but, just 
as importantly, to demonstrate how he or she applies 
what was learned. Various projects and portfolio 
assessments permit the student to show competency to 
enter a professional field — or to earn advancement in his 
or her field. 

Our findings are likely to revive an old debate from a 
new perspective. The debate concerns the quality of 
graduate education at a time when institutions are competing 
to increase the quantity of their students. In this context, 
some will argue that comprehensive assessments are 
becoming less rigorous — that the current situation 
represents a dumbing down of graduate education — 
reflecting the fact that we are scrambling to recruit (and 
graduate) students who would not have been accepted 
into our programs in the past. Others will argue that the 
alternatives outlined in this paper to the traditional written 
comprehensive examination demonstrate that universities 
have developed more reliable, realistic tools for measuring 
what a student has learned during an extended period 
rather than what he or she can memorize in a week of 
cramming and then promptly forget. Whatever the 
explanation and whatever its ultimate import, the 
comprehensive assessment is clearly in the process of 
undergoing significant, pervasive change. 

Implications 

Given the findings of this study, it would be helpful for 
the Tennessee Conference of Graduate Schools or some 
other organization — such as CGS — to conduct a follow 
up to Singleton’s 1 999 study, this time focusing on defining 
the various closure requirements in different disciplines 
and offering best practices. What, for example, constitutes 



for good portfolio assessment? In fine arts, when one 
institution says its students must present a capstone 
project and another institution says students must present 
an exhibit or recital to graduate, what distinction — if any — 
are the two institutions making about the nature of their 
respective culminating assessments? 

If an internship is used in lieu of a comprehensive 
exam, how does the capstone internship differ from 
internship courses which merely place students in so- 
called real-life situations? How does the culminating 
internship serve as an integrative summary of all the 
student has learned? 

A further consideration is this: if all these developments 
have occurred in the last 1 0 years or so, perhaps it is time 
to ask if the nature of the culminating experience is 
changing at the doctoral level as well. 

Editor’s Notes 

This edition of IR Applications is commendable from 
several perspectives, both methodological and focus. First, 
in terms of methodology, Deutsch and Nicholson work to 
bring together both the qualitative and the quantitative 
methodologies. In several of his works Quinn Patton argues 
that the strength of methodology is in the balance. The 
article also mentions the presence of a paradigm that 
guides the assessment methods in many graduate 
schools. Is it possible that the deans of these schools 
came from disciplines such as the natural sciences where 
the positivist or post-positivist paradigm dominates? The 
reader is encouraged to go to a discussion of paradigms 
such as that by Gupta in The Paradigm Dialog (Sage, 
1 991 , Newbury Park, CA) and consider the relevance of 
other paradigms such as the Constructivism or the Critical 
Theory approach to the validation of learning. 

The second commendable aspect of this research is 
the focus it takes. In some of my own previous research 
I have been forced to concur with the authors. There is a 
very serious lack of research on our graduate programs. 
All of our focus on learning outcomes as institutions 
seems to be on undergraduate experiences and learning. 
Even some of the professions, such as those in engineering, 
focus on undergraduate education. It would seem that 
institutions, in conjunction with professions, should establish 
and aggressively pursue a research agenda looking at 
graduate learning if we are to expect our graduate programs 
to be strengthened through management and assessment 
of learning. How should such an agenda be established? 
Who should do what? 

The third commendable aspect of this research is its 
existence. Several colleagues who had a vital interest in 
a topic seem to have established a systematic procedure 
and reached a point where at least initial conclusions can 
be drawn. They did not use sophisticated statistics and 
this does not seem to be a problem. Unfortunately they 
did not get a high return rate and this may also be 
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indicative of range of perspectives on the importance of 
their question - or it may be indicative of bad addresses. 
Their partnership with the Councii of Graduate Schoois 
wili hopefuily give the resuits visibiiity and wiii cause 
additionai assessments of graduate level expectancies. 

Endnote 

^ It should be noted that not all participants responded 
to all questions. This accounts for the differences in 
frequencies and percentages. The tables note the total 
number of responses to each question. 
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